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Abstract. Consider a dam model, L u PP er and L lower are upper and, respec- 
tively, lower levels, L = L u PP er — flower j g [ ar g e anc j jf ^ ne i eve j Q f wa ter is 
between these bounds, then the dam is said to be in a normal state. Passage 
across lower or upper levels leads to damage. Let J\ = jiL and J2 = j 2 L 
denote the damage costs per time unit of crossing the lower and, correspond- 
ingly, upper level where ji and j 2 are given real constants. It is assumed 
that input stream of water is described by a Poisson process, while the out- 
put stream is state dependent. Let Lt denote the level of water in time t, 
and c Lt denote the water cost at level Lt (L lower < U < L u W er ). Assum- 
ing that pi = limt-.oo P{L t = L lower }, p 2 = lim*^ P{L t > L"pp<=''} and 
qi = lim t ^ooP{Lt = i} (L lower < i < L u PP er ) exist, the aim of the paper 
is to choose the parameters of an output stream (specifically defined in the 
paper) minimizing the long-run expenses 



j = piJi +P2J2 + ^2 qiCi 



- L lo-wer + 1 



1. Introduction 

A large dam is defined by the parameters L lower and L upper , which are, re- 
spectively, the lower and upper levels of the dam. If the current level is be- 
tween these bounds, the dam is assumed to be in a normal state. The difference 
L = L upper — L lower is large, and this is the reason for calling the dam large. This 
property enables us to use asymptotic analysis as L — > 00 and solve easier different 
problems of optimal control than we would were than the dam is not large. 

Let L t denote the water level in time t. If L lower < L t < L upper , then the state 
of the dam is called normal. Passage across lower or upper level leads to damage. 
The costs per time unit of this damage is J\ = j\L and J 2 = J2L for lower and 
upper levels correspondingly, where ji and j'2 are given real constants. The water 
inflow is described by the Poisson process with rate A. In practice, this means that 
the arrival of water units is registered by counter at random instants t±, t2, ■ ■ ■ , and 
the times between consecutive instants are mutually independent and exponentially 
distributed with parameter A. 

The outflow of water is state-dependent as follows. If the level of water is between 
flower an( j j^upper^ ^ nen an interval between departures of units of water (inverse 
output flow) has the probability distribution function B\{x). If level of water 
exceeds L upper , then an inverse output flow has the probability distribution function 
B2(x). The probability distribution function B2{x) is assumed to obey the condition 
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J °° xdB2 (x) < j- If the level of water is L lower exactly, then output of water is 
frozen and it resumes again as soon as the level of water exceeds the level L lower . 

Let CL t denote the cost of water at level Lt. The sequence c, is assumed to be 
positive and non-increasing. The problem of the present paper is to choose the 
parameter xdBi (x) of the dam in the normal state minimizing the objective 
function 

L" ppcr 

(1.1) J=PlJl+P2J2+ ^ C **' 

where 

(1.2) pi = ]imP{L t = L lower }, 

t—>-oo 

(1.3) P 2 = lim P{L t > L upper }, 

t— >oo 

(1.4) qi = ]imP{L t = L lower + i},i=l,2,...,L. 

t — >oo 

In the queueing formulation, the level L lower is identified with an empty queue, and 
the dam model is the following queueing system with service depending on queue- 
length. If immediately before a service begins the queue-length exceeds the level 
L, then the customer is served by the probability distribution B 2 (x). Otherwise, 
the service time distribution is Bi{x). The value pi is the stationary probability 
of empty system, the value P2 is the stationary probability that a customer is 
served by probability distribution B%[x), and qi, i = 1, 2, . . . , L, are the stationary 
probabilities of the queue-length process, so Pi+P2+J2f=i <7* = !• (F° r the- described 
queueing system, the right-hand side limits in relations ||1.2|) - (|1.4|) do exist.) 

In our study, the parameter L increases indefinitely, and we deal with the series 
of queueing systems. The parameters above, such as pi, p2, Ji, Ji as well as other 
parameters are functions of L. The argument L will be often omitted in these 
functions. 

Similarly to [5] , it is assumed that the input parameter A and probability distri- 
bution function ^(x) are given, while the appropriate probability function B\(x) 
should be chosen from the specified parametric family of functions Bi(x, C). (Ac- 
tually, we deal with the family of probability distributions Bi (x) depending on two 
parameters 5 and L in series. Then the parametric family of distributions Bi(x, C) 
is described by the family of possible limits of SL as S — > and L — > oo.) 

The outflow rate associated with the probability distribution function B\(x) 
should be chosen such that the minimum of the objective function of (jl.ip is associ- 
ated with the choice of the parameter C resulting in the choice of the corresponding 
probability distribution function B\{x,C). 

The more particular problem, where the objective function has the form J = 
P\J\ +P2J2 (i.e. the water costs are not taken into account), has been studied in 
Abramov [5]. In this case the solution to the control problem is unique and has 
one of the following three forms. Denote P2 — A J °° xdB 2 (x) and p\ = pi(C) = 
A r o °° xdBx(x, C). Then, in the case ji — ]2 t"~~~~ , the optimal solution is pi = 1. In 
the case j\ > J2 jz 2 — , the optimal solution has the form pi = 1 + 5, where S(L) is a 
small positive parameter, and S(L)L — > C as L — > 00. In the case ji < 32 , the 
optimal strategy has the form p\ = 1—5, and 8{L)L — > C as L — > 00. The parameter 
C is a unique solution of a specific minimization problem precisely formulated in [5] . 
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It has been also shown in [5] that the solution to the control problem is insensitive 
to the type of probability distributions B\ (x) and B2 (x) . Specifically, it is expressed 
via the first moment of i?2(x) and the first two moments of B\(x). 

Following [I], we use the notation p\ t i = A ; J Q x l dBi(x), I =2,3. The exis- 
tence of a moment of the order corresponding to p\ t i will be specially assumed in 
formulations of statements corresponding to case studies. 

The problem studied in the present paper substantially distinguishes from that 
studied in [5]. Although the both control problem of the present paper and [5] 
are closely related, the new components of the present problem change the problem 
substantially. The problem of the present paper requires a much deepen and delicate 
analysis (for example, it is demonstrated in the next section that the asymptotic 
methods of the earlier paper [5] do not longer work here, and one should use more 
delicate techniques instead), and the main results on optimal control policies are 
deepen as well. 

Essential difficulty of the control problem in the present formulation is to prove 
a uniqueness of the optimal solution, while in the case of the particular problem of 
[5] , the uniqueness of the solution follows automatically from the explicit represen- 
tations of the functionals obtained there. 

It is assumed in the present paper that Cj is a non-increasing sequence. If the 
cost sequence Cj were an arbitrary bounded sequence, then a richer class of possible 
cases could be studied. However, in the case of arbitrary cost sequence, the solution 
need not be unique, and arbitrary costs Cj, say increasing in i, seem not to be useful 
and, therefore, are not considered here. The practical applications of the results 
obtained in the paper are not restricted by the area of water research. The results of 
the present paper are meaningful for many specific inventory and storage problems 
as well. Moreover, the model considered in this paper describes more realistically 
a production/realization process of large warehouses rather than an inflow/outflow 
process of large dams, because in many cases of a water inflow the seasonality is 
important. Nevertheless, similarly to [5], the problem formulation in this paper is 
given in terms of water research. 

More realistic models arising in practice assume that the probability distribu- 
tion function Bi(x) should also depend on i, i.e have representation B\ t i{x). The 
model of the present paper, where B\(x) is the same for all i, under appropriate 
additional information can approximate those more general models. Namely, one 
can suppose the stationary service time distribution B\(x) has the representation 
B\{x) = ^2 i=1 qiBi j i(x) (qi, i — 1,2, ...,L are the state probabilities), and the 
solution to the control problem for Bi(x) enables us to find then the approximate 
solutions to the control problem for B\^(x), i = 1, 2, . . . , L by using the Bayes rule. 

Similarly to the solution of the control problem of [5] , the solution of the present 
problem with extended criteria (jl.ip is related to the same class of solutions as in 
[5j. That is, it must be either p\ = 1 or one of two limits of p\ = 1 + 5, p± = 1 — 6 for 
positive small vanishing 5 as L increases indefinitely, and Lb — > C . The reason for 
this is that the penalties upon reaching upper or lower level are of order O(L) (i.e. 
increase to infinity as L — > 00), while the water costs arc assumed to be bounded as 
L tends to infinity, and although the water costs affect the solution of the control 
problem, this influence remains in the framework of the same class of solutions 
mentioned above. 

The following new questions are of special interest here. 
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1. What is the structure of an optimal solution? Is an optimal solution unique? 
The answer to these questions is the main result of the paper. The questions 

are answered by Theorem 13.31 We prove that a solution to the control problem 
does exist and unique, however there are some additional mild assumptions related 
to the class of probability distributions {B 1 (x)}. The proof of the existence and 
uniqueness of a solution is based on special techniques of Mathematical Analysis. 
Specifically, we use the known techniques of majorization inequalities [7], [5] in order 
to prove the monotonicity of specified functions. This property of monotonicity is 
then used to prove a uniqueness of a solution. 

2. Under what relation between ji, fa, P2, Ci (and maybe other parameters of 
the model) the optimal strategy is p% = 1? 

If the water costs are not taken into account, then the condition for p\ = 1 
is ji — fa x^p 2 • This result has been proved in [5]. It has a simple intuitive 
explanation and is a consequence of the well-known property of the stream of lost 
calls during a busy period of M/GI/l/n queues, under the assumption that the 
expected interarrival and service times are equal (see Abramov pQ as well as Righter 
[TO] or Wolff [16]). Taking into account the structure of water costs generally 
changes this condition for the aforementioned optimal solution pi = 1. We prove 
that the optimal solution pi = 1 is achieved under the condition ji < j-2j&^, and 
the equality in this relation holds if and only if the water costs are the same at all 
levels of water. This result is only the partial answer to the question. More exact 
answers can be obtained in particular cases, and one of them is the case of linearly 
decreasing costs as the level of water increases (for brevity, this case is called linear 
costs). In the case of linear costs we derive more exact and useful representations, 
which enable us to calculate numerically the relation between fa and fa to have 
finally the optimal solution p\ = 1. The relevant numerical results are provided for 
special values of the parameters of the model. 

The rest of the paper is organized as follows. In Section^ the asymptotic be- 
havior of the stationary probabilities is studied. This section is structured into four 
subsections. In Section \2. 11 some necessary results related to asymptotic properties 
of characteristic of the state-dependent queueing system are recalled for modelling 
the behavior of a large dam. In Section 12.21 elementary asymptotic properties of 
the state probabilities in the case p\ = 1 are established. The analysis of this case 
is based on application of a Tauberian theorem of Postnikov [9] for the convolution 
type recurrence relation. In Section 12.31 the behavior of the state probabilities is 
studied for the case p\ = 1 + S, where positive parameter S vanishes as L tends 
to infinity. The analysis of this case uses known results on asymptotic behavior 
of characteristics of the state-dependent queueing system obtained in Section 12.11 
Delicate asymptotic analysis is provided in Section [2~4l for the case p\ = 1—5, where 
positive parameter 6 vanishes as L tends to infinity. For the analysis of this case 
we involve the asymptotic result of Willmot [15] , which is also used in the present 
paper. All of the aforementioned asymptotic results of Section [2] are used to es- 
tablish asymptotic properties of special functionals. Then we solve the problem of 
minimization of these functionals. In Section [3J we solve the control problem. We 
prove existence and uniqueness of the solution under mild assumptions on the class 
of probability distributions {B\{x)}. We establish a structure of the solution and 
completely answer the question 1 posed in the introduction. In section |4] the case 
of linear costs is studied. We establish explicit representations for functionals of 
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the control problem and provide numerical answer to the question 2 posed in the 
introduction. 



2. Stationary probabilities of the state-dependent queueing system 
and their asymptotic behavior 

In this section, the explicit expressions are derived for the stationary probabil- 
ities, and their asymptotic behavior is studied. These results will be used in our 
further findings of the optimal solution. 

2.1. Preliminaries. We first recall some results of [5] and then develop them in 
order to obtain the explicit representations for the stationary probabilities that will 
be required in our further analysis. Let Tf,, vl, II denote correspondingly a busy 
period, the number of served customers during a busy period, and an idle period. 
Let v^ 1 and denote the number of customers served during a busy period by 
probability distribution functions B±(x) and B2(x) correspondingly, and let T^' 

(2) 

and T L denote the time spent for the service of customers during a busy period 
by the probability distribution functions B\(x) and B2(x) correspondingly. 

It was shown in [5] that the probabilities p\ and p2 can be expressed explicitly 
via Ei/1 1 ' , pi and P2 only. Namely, 

(2.1) Pl = 
and 

(2.2) p 2 = 



1 - Pi 



1 + ( Pl - p 2 )E4 1} ' 



P2 + P2(Pl - 1)E^[ 1) 

i + ( Pl - P2 )e4 1} ' 



Such kind of representations is convenient, because Ez/£^ satisfies the convolution 
type recurrence relation 

(2-3) E*£> = I>i-i + i /°V*^V(z), E,« = 1, 

where Ei4, denotes the expectation of the number of served customers during a 
busy period of the M/GI/l/n queue (n=0,l,. . . ). Recurrence relation l|2.3p is in 
turn a special case of the recurrence relations 



(2-4) Qn=J2< 



i-3 + l r 3 



with tq > 0, rj > for all j > 1, and ro + T\ + . .. = 1. The detailed theory of 
these recurrence relations can be found in Takacs [14] . Asymptotic behavior of Q n 
as n — > oo has been studied by Takacs [14] and Postnikov [9]. 

The stationary probabilities q,: can be obtained from the renewal arguments (e.g. 
Ross [H]). Namely, for i = 1, 2, . . . , L we have 

ET.W - ET-'^I 

(2 ' 5) * = ET L + ElI ' 
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where ET^ 1 ' denotes the expectation of a busy period of the M/GI/l/i queue 
(i=0,l,. . . ). The probabilities given by (|2.5p can be also written 

Ei/ 1} - Ei£.\ 

(2.6) ft=pi __ . 

Indeed, from Wald's equation [BJ, p.384 we have ET 2 (fc) = ^Ei/ fe) , fc = 1,2, and 
therefore the numerator of (|2.5p is rewritten 

(2.7) Elf-Elfl^E^-E^ 

Next, using the fact that the number of arrivals during a busy cycle coincides with 
the number of served customers during a busy period, from Wald's equation [BJ, 
p.384 we obtain 

(2.8) XET l + XEIl = Ev l . 
Therefore, (|2.6[) is the consequence of (|2.7|) and 



On the other hand, (|2.8|l can be rewritten 

AET L + AE/ L = AET L + 1 

(2.9) = A (eT^ + Elf } 

= PiE4 1} +p 2 E^ 2) 
From (|2.8p and (|2.9p we have the equation 

resulting in 



(2.io) E ,i 2) = j\ - ^e,« 



and, consequently by adding to the both sides of (|2.10p Ei/j^ , we obtain 
(2.11) E , £ = r i- + f^E,« 

1 — p2 1 — P2 



Substituting (|2.1ip for (|2.6p . we finally obtain 

M 1o1 Pl(l-P2) (1) tt, (1) \ -i r, 

(2.12) % = ( Ei/ - Ez^J , % = 1, 2, 

i + ( Pl - p 2 )e4 1} v y 

Comparison with (|2.ip enables us to rewrite (|2.12p in the other form 



.L. 



(2.13) q t = p lPl (Ez^ 



EI/&), i = l,2,. 



as well. 

In order to study the asymptotic behavior of the loss probabilities, let us recall 
the earlier results on asymptotic behavior of Ei/^ as L increases indefinitely (e.g. 
P]> [2], [I])- This asymptotic behavior can be obtained from the results of Takacs 
[14j . p. 22-23, on asymptotic behavior of recurrence relation (12. 4p as n — > oo. (The 
formulation of this Takacs theorem can be found in [5] as well.) 

If pi — 1 and pi t 2 < oo, then 



(2.14) lim 



Eu®_ 2 



L^oo L pi,2 
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and if pi > 1, then 



(2.15) lim 

L — > oo 



<p L (l + XB' 1 (X — \ip)) 



I -Pi 



where -Bi(s) is the Laplace-Stieljes transform of B\(x) 1 ip is the least in absolute 
value root of the functional equation z = B\(\ — Xz), and B[(s) denotes the deriv- 
ative of .Bi(s). 

We do not consider the case pi < 1, because it is not meaningful for our analysis. 
Only relations (|2.14p and (|2.15p are useful for the asymptotic analysis of the state 
probabilities, which is provided below. 

2.2. The case pi = 1. Asymptotic relation (|2.14[) is not enough in order to obtain 
an asymptotic behavior of the state probabilities. As it was mentioned above, this 
asymptotic relation (|2. 14[) is obtained from the aforementioned Takacs theorem 
[T4] , which has been also used in [5] for the analysis of the asymptotic behavior of 
Pi and P2 ■ The asymptotic analysis of the state probabilities is more delicate than 
that analysis of pi and P2, and asymptotic relation (|2.14[) is not enough for this 
purpose. Therefore, we have to use the more precise asymptotic estimation given 
by the Tauberian theorem of Postnikov [9] on the asymptotic behavior of recurrence 
sequence (|2.4j) . 



Lemma 2.1. (Postnikov [S], Sec. 25.) Let 71 = 5^ n=1 nr n = 1, 72 = S^°=2 n ( n ~ 
l)r„ < co, and r + T\ < 1. Let Q =/= be an arbitrarily given initial value of 
recurrence sequence (|2.4[) . Then as n — > co 

Qn+l -Qn = + O(l). 

72 

From this lemma we obtain the following statement. 
Corollary 2.2. For any j > 0, 

(2.16) Ez^jL- - Ei/^L-i = + as L -> °°- 

pi, 2 

Proof. In the case where rj = e~ Xx dBi(x), j — 0,1,..., the Tauberian 
condition ro + n < 1 of Lemma \2. II is always fulfilled, and the result of corollary 
follows directly from Lemma [2~T1 The proof of this fact uses the theory of analytic 
functions and has been provided in [3] (Theorem 4.6) and [2] (Theorem 3.3). Since 
this proof is elementary and short it is repeated here again. 
We must prove that for some Ao > the equality 

/•OO 

(2.17) / e~ Aoa: (l + Aoa;)dBi(a;) = 1 

Jo 

is not the case. If it is so, then only the inequality 



/>OC 

/ e~ Xx {1 + \x)dB 1 (x) <1 
Jo 



must hold. 

Indeed, J °° e~ Xx (1 + \x) dBi(x) is an analytic function in A, and therefore, 
according to the theorem on the maximum module of an analytic function, equality 
(I2.17[) must hold for all Ao > 0. This means, that (|2.17[) is valid if and only if 
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/ o °°e- Aoa; ^^dBi(a;)=0 for all j > 2 and A > 0. In this case the Laplace- 
Stieltjes transform -Bi(A) must be a linear function in A, i.e. £?i(A) = do + d±X, do 
and d\ are some constants. However, since |Bi(A)| < 1, we have do — 1 and d\ = 0. 
This is a trivial case where B\{x) is concentrated in point 0, and therefore it is not 
a probability distribution function having a positive mean. Thus (|2. 1T[) is not the 
case, and the statement of the corollary follows. □ 

From (|2.16[) and (|2.ip . (|2.13|) we have the following lemma. 
Lemma 2.3. In the case p\ = 1 and p2 < oo for any j > we have 
(2.18) lim LqL-j = 1. 

L — >oo 

Proof. Asymptotic relation (|2~18|) follows immediately from (|2~T1l . (f2~13|) and (f2TT6]l . 

□ 



Combining the result obtained in [5] and Lemma 12.31 we have the asymptotic 
result for the functional J = J\p\ + J2P2 + Si=i Qi c i- 

Proposition 2.4. In the case p\ = 1 and ^2 < cxd we have 

/0 im ,. ,frs ■ Pi, 2 . P2 Pi, 2 , * 

(2.19) hm J(L) = ji— +j 2 : 



i-»oo 2 1 — P2 



»'/) 



ere 



1 L 

c* = lim — y a 

1=1 



Proof. The first two term of the right-hand side of (|2.19p easily follow from (|2.ip , 
(|2.2p and asymptotic representation (|2. 14[) . The last term of the right-hand side of 
(I2.19[) follows from Lemma T2.3[ since 



l ^ L 

lim qiC t = lim — c t 

i=l i=l 



Notice, that the representation for the first two terms of the right-hand side of (|2.19[) 
has a relation to the particular case of the dam model not taking into account the 
water costs and been obtained in [5]. □ 



Remark 2.5. The statement of Proposition [2~ll confirms that the class of the possible 
solutions of the control problem is the same as that for the particular problem 
studied in [5], since according to Proposition ^. 41 the optimal value of the functional 
J is finite (i.e. does not increase to infinity as L increases indefinitely) and cannot 
exceed the right-hand side of (|2.19[) . 

2.3. The case px = 1 + 5, 6 > 0. In the case p\ = 1 + 5 we have the following. 

Theorem 2.6. Assume that p\ = 1 + 6, 6 > 0, and LS — > C > as 5 — » 

and L —> 00. Assume that pi^(L) is a bounded sequence, and there exists p\^ = 
liniL^oo pi,2(L). Then, for any j > 
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Proof. Expanding first (|2.15|) for large L, we have 



(2.21) 



(i) 



1 + XB'^X — Xcp) 



I -Pi 



o(l) 



From (|2.21[) for large L we have 



(2.22) E^V E ^Vi 



(1 - tp)<p> 



1 + XB[(X - Xip) 



o(l). 



Next, under the condition of the theorem we have the following expansion 



(2.23) 



<p = 1 - M- + 0{8 2 ). 

Pl,2 



(The details of this expansion can be found in [15], p. 326 or [3], p. 21. See also the 
proof of a similar fact in Lemma \2 . 91 below . ) Then, taking into account (|2.23p . we 
also have 



(2.24) 



1 + \B[(\ - Xtp) = 5 + 0(6 2 ). 



Next, taking into account asymptotic expansions (I2.23[) and (I2.24p from (|2.2ip and 
([2722]) we have 



(2.25) 

and for any j = 0, 1, . 
(2.26) 



(i) 



3 2C/p lj2 _ 1 



1 



+ 0(1), 
26 \ 3 2 



[l + o(l)]. 



a 2C/p 1>; 

' Pi, 2 J Pl,2 

Now, using explicit representation (|2.12[) . for any 7 = 0,1... we obtain the desired 
asymptotic relation 



3 2C/pi 



9L-j 



e 2C/pi, 2 _ 1 



1 



26 Y 26 



Pi, 2/ Pl.2 



[1+0(1)]. 



The theorem is proved. 

Using the result of [5] we have the following. 



□ 



Proposition 2.7. Under the assumptions of Theorem ! 2. 6\ denote the objective func- 
tion J by J u PP er . We have the following representation 

p2C/pi,2 



(2.27) jnpper _q 

where 
(2.28) 



1 



3i 



e 2C/p 1 , 2 _ 1 _ p2) (e2C/pi,2 _ i) 

2C 



p2er 



2(7 p 2C7 /pi.2 i _ 

upper = = Um C ( ! 

Pi, 2 e 2C lP 1 -' 2 - 1 i^oo L 



Pl,2L 



and Cl(z) = ^2^=0 c L~jZ° is a backward generating cost function. 
Proof. The representation for the term 



C 



1 



.71- 



2C/p 1(2 _ ! 1 -? 2 (! _ p 2 )( e 2C/?i,2 _ 1) 



2C/p 1(2 
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is known from [5]. The new term in (|2.27|) . which takes into account the water 
costs, is c upper . Therefore, keeping in mind representation (|2.20[) . for this term we 
obtain: 



L-l L-l 



x- - , x- - e 2C/ Pl ,2 f 26 L V 2SL 

x qL-jCL-, = lim > i 



and representation (|2.28|) follows. □ 

2.4. The case p% = 1 — #, 8 > 0. The study of this case is more delicate and based 
on a special analysis. Our additional assumption here is that the class of probability 
distribution functions {Bi(x)} is given such that there exists a unique root r > 1 
of the equation 

(2.29) z = B(X-Xz), 

and there exists the first derivative B'(X — At). 

Under the assumption that p\ < 1 the root of (|2.29[) is not necessarily exists. 
Such type of condition has been considered by Willmot [15] to obtain the asymptotic 
behavior for high queue- level probabilities in stationary M/GI/1 queues. Denote 

the stationary probabilities of M/GI/1 queues by qi[M/GI/l], i — 0, 1, There 

was shown in [15j that 

(2.30) qAM/GI/1] = ~ ~ T > [1 + o(l)], asz-^oo. 

t 1 [1 + \B'{\- At)] 

On the other hand, according to the Pollaczek-Khintchinc formula (e.g. Takacs 
[13], p. 242) qi[M/GI/l] can be represented explicitly 

(2.31) qAM/GI/1] = (1 - Pl ) (E^ - Ei/W ) , i = 1,2, . . . . 
(Representation (|2.31[) can be easily checked, since it easily follows from (|2.3p that 

(2.32) fWy= ^{X-Xz) 

Bx(X-Xz)-z 

and multiplication of the right-hand side of (|2.32p by (1 — pi)(l — z) leads to the 
well-known Pollaczek-Khintchinc formula.) 

From (|2.30p and (|2.31[) we also have the following asymptotic proportion. For 
large L and any j > 

E^.-Ez.W. x 

(2.33) —^5 = 1 + 1 . 

Now we formulate and prove a theorem on asymptotic behavior of stationary 
probabilities in the case p± = 1 — 6. In this theorem we assume that the class 
of probability distributions {Bi(x)} is defined according to the above convention. 
In the case where pi = 1 — 6, 8 > and vanishing 8 as L increases indefinitely, 
this means that there exists eo > (small enough) such that for all < e < eo, 
the above family of probability distribution functions i?i. c (x) (depending now on 
parameter e) satisfies the following properties. Let i?i je (s) denote the Laplace- 
Stieltjes transform of Bi e (x). We assume that any Bi >e (s) is an analytic function 
in a small neighborhood of zero, and 

(2.34) B[ J~Xe) < oo. 
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Property (|2.34|) is required for the existence of the probabilities qi. Relation (|2.30p 
contains the term J5'(A — At), and for r = 1 + e this term should be finite. Choice 
of the small parameter e is closely connected with that choice of parameter 5 in the 
theorem below. 

Theorem 2.8. Assume that the class of probability distribution functions {Bi(x)} 
is defined according to the above convention and satisfies (|2.34[) . p\ = 1 — 5, 5 > 0, 
and L5 — > C > 0, as 5 — > and L — > oo. Assume that pi.3 = pi^(L) is a bounded 
sequence, and there exists pi^ = hin^^oo p\^(T\ Then, 

2(5 1 / 25 \ j , 

(2 ' 35) to - = ^-^7^l( 1 + ^J + 

/or any j > 0. 

Proof. We start the proof from the following auxiliary result, similar to that used 
to prove Theorem 12.61 

Lemma 2.9. Under the assumptions of Theorem \2.8l the following asymptotic 
representation holds 

(2.36) t = 1 + ^-+0(5 2 ). 

Pl,2 



Proof of Lemma AzM The proof of this lemma is completely similar to the proof of 
the result in [T2], p. 326 mentioned in the proof of Theorem l2.6l fsee relation (|2.23p . 
According to the above convention, the equation z = B(X — Xz) has a unique 
solution r > 1. Clearly, the root approaches 1 as 5 vanishes, because otherwise we 
would have a contradiction with Corollary |2.2l Therefore, by the Taylor expansion 
of this equation around the point z = 1, we have 

z = 1 - (1 - 5){1 - z) + ^(1 - zf + 0(1 - zf. 

Ignoring the last term 0(1 — z) 3 of this expansion, we have the quadratic equation, 
and the two solutions of the equation are z = 1 and z = 1 + ■ Therefore, 

Pl,2 

t = \ + ^ + 0{5 2 ), 
Pi, 2 

and the lemma is therefore proved. □ 

Let us continue the proof of the theorem. From (|2.33[) and (|2.36[) of Lemma 
we have: 

(2.37) qL _ j = qL ( 1 + lLY [l + o(l)}. 



Pi, 2 



Taking into account that 
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from the normalization condition pi + P2 + Ej=i <7j = l an ^ the fact that the both 
Pi and pi have the order 0(S), we obtain: 

25 1 



(2.38) 



1L 



[1 + 0(1)]- 



Pi, 2 e 2C /'' 1 - 2 - 1 
The desired statement of the theorem follows from (|2 . 38[) . 

Using the result of [5] we have the following. 

Proposition 2.10. Under the assumptions of Theorem 
function J by J lower . We have the following representation 



□ 



denote the objective 



(2.39) 
where 
(2.40) 



J 



= C 



h^ /2C +j2 T ^(^ /2C -l) 
1 - P2 V / 



dower 



2C 



1 



lim i C L [1 



2C 



Pi, 2 e 20 /^ 1 - 2 - 1 l^oo L \ piaL J ' 
and C{z) = ^2^ = q CL-jZ-* is a backward generating cost function. 
Proof. The representation for the term 



C 



l-p2\ > 



is known from [5]. The new term in ()2.39j) . which takes into account the water 
costs, is c lower . Keeping in mind representation (|2.35[) . for this term we obtain: 



L-l 



L-l 



lim Ql-jCl-j = lim ^ c L 



3=0 



L — *oo 



3=0 



2SL 1 
i ' p 12 L ' e^c/pLi _ i 



25L 



and representation (|2.40|) follows. 



□ 



3. Solution of the control problem and its properties 



In this section we discuss the solution to the control problem and study its 
properties. The functional J upper and J lower are given by (j2~2Tj) and (|2~3"9"|) . The 
last terms of these functionals are 

L-l 



(3.1) 

and 

(3.2) 



upper 



2C 



3 2C/pi, 2 



Pi,2 e 



2C/pi, 2 



- lim — Cl-4 ( 1 



3=0 



L-l 



25_^ ' 



2C 1 ,. 1 / 25 

~ — • „„ ,~ urn — > CL-j 1 + — — 

Pi, 2 e 2C /P^< 2 - 1 L^oo L ^— ' V pi 2 



correspondingly, where C = liniL^oo L5. 

Now we give other representations for these terms. Denote 



(3.3) 



m = lim 

L—>-oo 



E^O 1 C L-3 ( 1 



2C_ 

Pl,2 L 



( i - 



2C 

Pl,2^ 
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and 



Y^l-i 
2^7=0 c L-j 



sj=0 

(3.4) 77(C) = lim 



2C 



Since {c{\ is a bounded sequence, then the both limits of (|3.3j) and (|3.4j) do exist. 
We have the following lemma. 

Lemma 3.1. We have 

(3.5) c upper = V-(C), 
and 

(3.6) c lower = r)(C). 

Proof. From (|3.3[) and (|3.4p we correspondingly have the representations 



and 



i=o V ^ / J= o v pi <^ 



L 1 2C V ,. 1 ^ / 2C 



( 3 - 8 ) & 7 E 1 + S J = « & 7 E 1 



L— >oo L ^ — ' V pi 7 L^oo L ^ — ' V pi 2L 

The desired results follow by direct transformations of the corresponding right-hand 
sides of dH]) and (j3~gj) . 

Prove (|3.5p . For the right-hand side of (|3.7p we have 

2C Y_./.^Al „-2C/p!. 2 A Pi,2 



L— >oo L ^ — ' \ j9i oL J \ } 2C 



(3.9) lim 7 E ( 1 - ) = ^ i 1 

On the other hand, from (13.111 we have 



e 



,3.10, (l - 2g . Jja I (l - ^ 

j=0 x r ' 

Hence, from (|3T| . (|3~9| and (|3~T0j) we obtain ([321). The P r001 of (EH) is completely 
analogous and uses representations (|3.2p and (|3.8p . □ 

The next lemma establishes the main properties of functions ^){C) and 77(C). 

Lemma 3.2. The function ip(C) is a non- increasing function, and its maximum 
is i/j(0) = c* . The function 77(C) is an increasing function, and its minimum is 
7/(0) = c*. (Recall that c* = liniL^oo ~ Ylf=i °i * s defined in Proposition \2.4\ ) 

Proof. Let us first prove that 0(0) = c* is a maximum of "0(C). For this purpose 
we use the following well-known inequality (e.g. Hardy, Littlewood and Polya [7] or 
Marschall and Olkin 8 ). Let {«„} and {b n } be arbitrary sequences, one of them 
is increasing and another decreasing. Then for any finite sum we have 

/ i i 

(3.11) dnbn < 7 / °n / &n- 

n—1 n—1 n—1 
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Applying inequality (|3.11|) to finite sums of the left-hand side of ()3.7|) and passing 
to limit as L — > oo, we have 

1 1 ' 2C 



3 -=0 V FL ' Z 



,. 1 ^ 1 2C \ 

3.12 < hm - > C£_j lim -) 1 - ^— ■ 



3=0 3=0 
L-l 



2c 1 

j=0 V 

Then, comparing f|3 . T|) with (|3.12[) enables us to conclude, 

m = c* > ^(c), 

i.e. ^>(0) = c* is the maximum value of t()(C). 

Prove now, that ip(C) is a non-increasing function, i.e. for any nonnegative 
d < C we have V(C) < ^(Ci). 

To prove this note, that for small positive <5i and 62 we have (I-61-62) = (l-5i ) 
(I-S2) + 0(6162)- Using this idea, one can prove the monotonicity of ip(C) by 
replacing 

Pi.zl) ( Pi,2W \ Pi,2i ) +< ^(i 2 )' ^ > 
in the above asymptotic relations for large L. Indeed, notice that 

L-l 



i im ir 1 

(3.13) 



2Ci Y Z' 2C-2Ci XJ 



l-^oo L t - ^ \ P1.2LJ \ Pi,2L 

L-l / n ^ \ ?' -, L-l 



lim - V 1 - ^-L lim - y 1 — - - 



L^oo £ V p\ 2L I L^oo _L ^ — ^ \ fli 2^ 



Therefore, for any non-decreasing sequence a 



L ' 2Ci \ '' / 2C-2Ci XJ 



lim — a, f 1 — 



. . 1 - 

L^oo L ^ \ Pi, 2^/ V Pl,2^ 
(3.14) J 

,. 1 £pj / 2Ci V l^A 2C-2Ci 
< lim - > a , 1 - — — lim - > 1 

Indeed, assume for contrary that 

i^i^J V pi,2^y V pi,2^ 



(3.15) 



> 



11 ' 2d V 1 ^4 / 2C-2CV 



L— >oc L — ' \ jOi oL J L—fOo L — 4 V pi 9-L 

j=0 x 7 j=0 x ^ 1,z 
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Then, applying the inequality (|3.11j) to the right-hand side of (|3.15j) . we obtain: 

. s ,. 1 ^ v 1 v^V 2d V ,. 2C-2CiY 
3.16 < lim - V 0i Um - V 1 - lim - V 1 „ 

= lim — a,- lim — ( 1 - J — - j . 

i=o j=o v 7 

Comparison of the last obtained term with the left-hand side of (|3.15[) enables us 
to write: 




The contradiction with the basic inequality p. lip proves (|3.14p . 

Taking into account (|3.13p and (|3. 14)1 the extended version of (|3.12p after appli- 
cation (|3.1ip now looks 

. lim iv eL ,,Yi-jgLyYi- 2c L- 2c 'y 

(3.17) J 

< Bm I £ f , - «lV Bm I £ (j _ 3£^ay 

1 1} l^ooL^\ hfiL J L^Lj^\ p h2 L J 
On the other hand, the right-hand side of (|3.7p can be rewritten 

,3,8, .^ta '£(1 ^y^^y 

I^oo I ^ \ Pi,2 L J V. Pl,2i / 

The last equality in (|3. 18|) is an application of (I3.13p . From (|3.17p and 13 . 1 8[) we 
finally obtain V'(Ci) > "0(C) f° r an y positive C\ < C. The first statement of Lemma 
13.21 is proved. The proof of the second statement of this lemma is similar. □ 

We are ready now to formulate and prove a main theorem on optimal control of 
the dam model considered in the present paper. 
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Theorem 3.3. The solution to the control problem is pi — 1 if and only if the 
minimum of the both functionals J u PP er and J lower is achieved for C = 0. In this 
case the minimum of the functional J is given by (|2.19p . Otherwise, there can be 
one of the following two cases for the solution to the control problem. 

(1) If the minimum of J upper is achieved for C = 0, then the minimum of J lower 
must be achieved for some positive value C — C_. Then the solution to the control 
problem is achieved for p\ = 1 — 8, where 8 > vanishes such that SL — > C_ as 
L —> oo. 

(2) If the minimum of J lower is achieved for C — 0, then the minimum of J upper 
must be achieved for some positive value C — C . Then the solution to the control 
problem is achieved for p\ = 1 + 5, where 8 > vanishes such that SL — > C as 
L — > oo. 

The minimum of the functionals J u PP er or J lower is defined by their differenti- 
ating and then equating these derivatives to zero. 

Proof. Note first, that there is a unique solution of the control problem considered 
in this paper. Indeed, in the case where the water costs are not taken into account, 
the existence of a unique solution of the control problem follows from the main 
result of [5] . In the case of the dam model of this paper the only difference is in 
presence of the functions c upper and c lower in the functionals J upper and J lower 
correspondingly. According to Lemma [3.11 c upper — tp(C) and c lower = r/(C), and 
according to Lemma l3~2"l the function ip(C) is non- increasing in C, while the function 
77(C) is a non-decreasing in C, and V'(O) = ??(0) = c*. Therefore, there is a unique 
solution of the control problem considered in the present paper as well. 

In the case where the both minima of J u PP er and J lower are achieved in C = 0, 
then each of these minima is equal to the right-hand side of (|2.19p , and the minimum 
of the objective function J is achieved for p\ = 1. 

If the minimum of j lower i s achieved for C — C_, then j lower i s less than the 
right-hand side of (|2.19[) . and, because of uniqueness of the solution, the minimum 
of J u PP er must be achieved for C = 0. In this case the minimum of the objective 
function J is achieved for p\ = 1 — 5, 5 > vanishes as L — > 00, and L5 — > C_. 

In the opposite case, if the minimum of J u PP er is achieved for C — C, then 
jupper j g j egg th an the right-hand side of (|2.19j) . and the minimum of J lower must 
be achieved for C — 0. In this case the minimum of objective function J is achieved 
for pi = 1 + 5, 8 > vanishes as L — > 00, and L8 — > C. □ 

From Theorem l3.3l we have the following evident property of the optimal control. 

Corollary 3.4. The solution of the control problem can be pi = 1 only in the case 
ji < Ja i- 2 ■ Specifically, the equality is achieved only for Ci = c, i = 1, 2, . . . , L, 
where c is an any positive constant. 

Although Corollary [3T4] provides a partial answer to the question 2 posed in the 
introduction, the answer is not satisfactory, because it is an evident extension of 
the result of [5] . A more constructive answer the question 2 of the introduction is 
obtained for the special case considered in the next section. 

4. Example of linear costs 
In this section we study an example related to the case of linear costs. 
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Assume that c\ and cl < c\ are given. Then the assumption that the costs are 
linear means, that 

(4.1) c t = ci - 1— -(ci -c L ), i = l,2,...,L. 



It is assumed that as L is changed, the costs are recalculated as follows. The 
first and last values of the cost ci and cl remains the same. Other costs in the 
intermediate points are recalculated according to (|4.ip . 

Therefore, to avoid confusing with the appearance of the index L for the fixed 
(unchangeable) values of cost ci and cl, we use the other notation: c\ = c and 
cl = c. Then (|4.ip has the form 

(4.2) Ci = c-l_i(c-c), i = l,2,...,L. 

In the following we shall also use the inverse form of (|4.2p . Namely, 



(4.3) c L ^ = c+—!—-(c-c), i = 0,l,...,L-l. 

Li — 1 



Apparently, 



(4-4) c* = ——. 



For c upper we have 



c u PP er = -0(C) = lim 



Z^7=o 1 



(4.5) _ ]_ Ej=o J 

= c + (c — c) lim 



L-l N 



1 



Pi, 2^ 



2C* 



. /_ A Pl,2 Pl,2 

c+ (c-c) — 



+ e 2 C/Pi.2 _ 1 



2C e 2C /p^ - 1 

For example, as C converges to zero in (|4.5j) . then c upper converges to c+ |(c— c) 
c* . This is in agreement with the statement of Proposition 12.41 
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c lower = ^ = Hm 



Elo 1 [c+-r^-(c~c) 



L-l J \ pifiL 



2C 



2^=0 



\r^L — l . ( 2C 



- c + (c — c) lim 



2^.7=0 



Pia 1 



c+(c-c) — 



2C 1 - e-^/ P i,2 

Again, as C converges to zero in (14.61) . then c lower converges to c+ |(c- c) = c* . 
Again, we arrive to an agreement with the statement of Proposition 12.41 

Let us now discuss the question 2 posed in the introduction. We cannot give 
the explicit solution because the calculations are very routine and cumbersome. 
However, we explain the way of the solution of this problem and find a numerical 
result. 

Following Corollary 13.41 take first ji — J2 \ zr~ 2 ■ Clearly, that for these relation 
between parameters ji and j'2 the minimum of J lower must be achieved for C = 0, 
while the minimum of J u PP er must be achieved for a positive C. Now, keeping ji 
fixed assume that ji increases. Then, the problem is to find the value for parameter 
j2 such that the value C corresponding to the minimization problem of J u PP er 
reaches the point 0. 

In our example we take ji = 1, pa = hi 0. = 1, c = 2, jo 1;2 = 1- In the table below 
we outline some values j2 and the corresponding value C for optimal solution of 
functional J u PP er . It is seen from the table that the optimal value is achieved in 
the case j'2 ~ 1.34. Therefore, in the present example j\ = 1 and j"X ~ 1-34 lead to 
the optimal solution p\ = \. 
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